Efficient quantification of experimental evidence against local realism 
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To quantify the evidence against local realism in an experiment, one can choose a test statistic to 
measure the amount of violation of local realism. It is desirable to bound the probability, according 
to local realism, of obtaining a test statistic at least as extreme as that observed. We describe an 
efficient protocol for computing such a bound from any set of Bell inequalities for any number of 
parties, measurement settings or outcomes. The bound depends on the choice and number of Bell 
inequalities, and more inequalities make the bound asymptotically tighter. We find that even trivial 
Bell inequalities such as those derived from no-signaling conditions can improve the tightness of the 
bound. 
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Theories designed according to "local realism" (LR) 
include a set of hidden variables, which if known would 
predict all measurement results; however, the values of 
the hidden variables cannot be influenced by spacelike- 
separated events. In 1964 Bell constructed an inequality 
satisfied by all correlations accessible by LR and showed 
that correlations between spacelike-separated measure- 
ments on two quantum systems can violate this inequal- 
ity pp. Since then, many experimental tests showing Bell- 
inequality violations have been performed (see Ref. [5] 
for a review). The importance of such a test is twofold. 
First, it shows that local realistic (LR) descriptions of 
bipartite quantum systems do not always exist. Second, 
it supports quantum information tasks such as quantum 
key distribution [3H5] and randomness generation [SI [7] . 

To test a Bell inequality in an experiment, one needs 
to estimate the probabilities of various outcomes from 
a hnite number N of measurements. Due to uncertain- 
ties in the estimated probabilities, it is conventional to 
present the violation of LR in terms of the number of 
experimental standard deviations of violation of a Bell 
inequality. While this provides the precision with which 
a Bell-inequality violation is measured, it is not a valid 
indicator of the strength of experimental evidence against 
LR [5] . For the latter, one needs to take into account the 
possibility that N data points generated by an LR model 
can violate a Bell inequality due to statistical fluctuations 
in finite samples. This possibility can be formalized in 
statistics via a p- value for the hypothesis test of LR. We 
define a p- value as the maximum probability according to 
LR of obtaining a test statistic, such as a Bell-inequality 
violation, at least as extreme as that observed. Thus, a 
small p- value means that the observed data are significant 
for rejecting LR. Upper bounds of p-values for specified 
test statistics are required for precise statements on ex- 
perimental evidence against LR. Such bounds not only 
help to reliably demonstrate violations of LR, but also 
help to prove the security of quantum key distribution or 
certify the generation of genuine randomness. 



There are two available protocols that compute upper 
bounds of p-values. One is the martingale-based proto- 
col but the bounds computed, such as the bound 
in Ref. \§\ , are not tight [5] . The other is the prediction- 
based-ratio (PBR) protocol [8], which computes tighter 
bounds. Specifically, the latter bounds are asymptoti- 
cally tight with respect to N, if the prepared quantum 
states and measurement settings do not vary in time. 
While the PBR protocol is practical for many standard 
configurations, it is inefficient with respect to the num- 
ber of parties per test, settings per party, and outcomes 
per setting. The reason is that it requires computing es- 
timates of the experimental probability distribution and 
the associated optimal LR model. These estimates are 
difficult to find when there are many parties, settings, 
or outcomes. Extreme examples are provided by config- 
urations involving continuous variables, where the PBR 
protocol cannot be directly applied. Here, we propose 
a simplified PBR protocol to efficiently compute high- 
quality p-value bounds for all configurations. 

The simplified PBR protocol has at least three advan- 
tages over other protocols. First, the p- value bounds 
computed are as good as and typically better than those 
obtained by the martingale-based protocol. Second, mul- 
tiple Bell inequalities can be taken into consideration at 
once in a statistically rigorous way. Thus we can obtain 
high-quality p-value bounds even when we cannot deter- 
mine beforehand which inequality will work best. Third, 
this protocol can be applied to any test with linear wit- 
nesses, such as entanglement detection [HJ[H], without 
a full analysis of the relevant probability space. 

Preliminaries. — An experimental test of LR involves a 
number of trials. At each trial, each of a number of spa- 
tially separated parties performs a local measurement, 
where the setting is chosen randomly from a fixed set. 
Conventionally, at the end of the experiment, a prede- 
termined Bell inequality is tested using the results from 
the trials. For example, if there are two parties and 
each party has two measurements with outcomes ±1, the 
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Clauser-Horne-Shimony-Holt (CHSH) inequality [12] 

E(A 1 B 1 ) + E(A 1 B 2 ) + E(A 2 B 1 ) - E(A 2 B 2 ) < 2 (1) 

can be tested, where E(AiBj) with i,j E {1,2} is the 
correlation between measurements A t and Bj. 

To statistically quantify the evidence against LR, we 
express a Bell inequality as an upper bound on the ex- 
pectation of a function of a trial result. That is, we 
write a Bell inequality in the form (I(X)) < B, where 
/ is a real-valued function, called a Bell function, and 
X is the random variable from which a trial result x 
is sampled. The result x consists of the measurement- 
setting choices made by all parties and the outcomes 
of these measurements. To write a Bell inequality in 
the above form, the probability distribution of the joint 
measurement-settings is assumed to be known and fixed 
before an experiment. There is no loss of generality in 
assuming this, as explained in Refs. [SUlOj. For example, 
for the CHSH inequality ([T]), a trial result x consists of 
setting choices i,j and outcomes a%, bj, and we can write 
^chsh(£) = 4(1 — 2<5i ) 2<Sj 1 2)fli&j and B = 2, where we have 
assumed that the joint-setting distribution is uniform. 

As explained in the introduction, we quantify the 
strength of experimental evidence against LR by means 
of a p- value. A p-value is associated with a test statistic 
T that is a function of the sequence of trial results. If N 
is the total number of trials, the corresponding sequence 
of results is denoted by x = {x\,...,xn). As is con- 
ventional, we distinguish between the sequence of results 
and the sequence of random variables X = {X\, . . . , Xn) 
giving rise to these results. The exact p- value pat is de- 
fined as the maximum of the probabilities of the events 
T(Xlr) > T(x) over all random- variable sequences Xlr, 
distributed according to LR models. That is 

Pn = maxProb LR (T(X LR ) > T(x)). (2) 

Due to the difficulty of determining worst-case tail proba- 
bilities of typical test statistics, we can usually determine 
only upper bounds of exact p- values. Thus, for the re- 
mainder of the paper, the term "p-value" refers to any 
valid upper-bound on the exact p-value. For the proto- 
cols discussed below, if the trial results are independent 
and identically distributed according to a distribution 
that violates LR, the p-values computed decrease to 
exponentially as N ~ > oo. We can therefore compare dif- 
ferent protocols' performances in a test of LR according 
to the confidence-gain rate defined by 

G=-hm 1 ° g ^, (3) 

where p^™'' is the p-value computed by a protocol. 
Higher gain rates imply better protocol performance. 
Each protocol discussed below works even under mem- 



ory effects [13] , that is, even when the prepared quan- 
tum state, measurement settings, and relevant LR mod- 
els vary arbitrarily with time. 

PBR protocols. — The test statistic used by a PBR pro- 
tocol is based on nonnegative functions R n to be applied 
to the n'th trial result x n and satisfying (R n (X)) < 1 for 
X distributed according to any LR model. Thus, each 
R n is a nonnegative Bell function for the Bell inequality 
(R n (X)) < 1. The function R n is constructed before ob- 
serving the n'th trial result x n . Its construction can use 
information from previous trials and typically requires 
predicting the distribution of X n . Thus, R n is referred to 
as a prediction-based ratio (PBR). A PBR protocol com- 
putes a test statistic according to T(x) = Y\. n =i Rn( x n)- 
To obtain a p-value for T(x) it suffices to observe that 
by construction T is nonnegative and (T(Xlr)) < 1, so 
that by Markov's inequality we can compute a p-value 
according to 

p£ BR) =min(l/T(x),l). (4) 

See Ref. [5] for further details. Different PBR protocols 
are characterized by how they choose the PBR R n for 
each n. 

Full PBR protocol.— For the full PBR protocol [3], 
R„ is chosen so as to optimize the expected confidence- 
gain rate given previous trial results. For this optimiza- 
tion, the protocol assumes that A n 's distribution is the 
same as that from which the trial results x%, . . . , x n -\ 
were sampled, and that these samples are independent. 
Whether or not these assumptions actually hold affects 
only the quality of the p-value computed, but not its va- 
lidity. Given these assumptions, R n is computed in two 
steps. The first is to make an estimate of the experimen- 
tal probability distribution q(x) = ProbQM^n = %), 
and the second is to determine the probability distri- 
bution p(x) = ProbLR,(^LR = x) according to the LR 
model that minimizes the Kullback-Leibler (KL) diver- 
gence from the estimate q [TS]. The protocol then sets 
the next PBR to R n (x n ) = q(x n )/p(x n ). Details about 
this protocol and the proof that this R n satisfies the con- 
ditions on a PBR are in Ref. [8]. 

While the implementation of the full PBR protocol is 
practical for typical experimental tests of LR, its com- 
plexity is not well-behaved as the number of parties, 
settings, or outcomes increases. In particular, on each 
PBR computation, it needs to optimize a convex objec- 
tive function over the convex space of all LR models, 
where the evaluation of the objective function involves a 
sum over all possible setting and outcome combinations 
of the parties. In contrast, the simplified PBR protocol 
introduced next requires an optimization over a small- 
dimensional convex space of a convex objective function 
that can be written as a sum of at most (N — 1) terms. 

Simplified PBR protocol. — The simplified PBR proto- 
col chooses the PBRs from the convex combinations of 
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Bell functions that are derived from a given set of Bell 
inequalities. To ensure that a convex combination is a 
PBR, the Bell functions first need to be standardized 
so that they are nonnegative and have expectations at 
most 1 for any LR model. Any Bell function that is 
lower-bounded has such a standardized form. In partic- 
ular, if (I(X)} < B is a Bell inequality and I(x) > b 
for all x, then r(x) = (I(x) — b)/(B — b) is standard- 
ized. Note that, as a constraint on the distribution of 
X, (r(X)) < 1 is equivalent to (I(X)) < B. Given 
Bell inequalities (j( m ) (X)) < BW where is lower- 
bounded and m — 1,2, ...,M, we can construct the 
corresponding standardized Bell functions A m \ We de- 
fine r = (r^, . . . ,r^ M '). The simplified PBR protocol 
chooses the PBR R n from among the convex combina- 
tions 



r = ^2 u n 



(5) 



where u m > and X) m Wm = 1- Our implementation 
always includes the trivial Bell function r' 1 ) = 1. This 
ensures that the set of convex combinations is at least 
one-dimensional and that the confidence-gain rate is at 
least as high as that achieved by the martingale protocol. 

Like the full PBR protocol, the simplified PBR pro- 
tocol aims to optimize the expected confidence-gain 
rate given previous trial results, under the assump- 
tion that the distribution of X n is the same as the 
empirical-frequency distribution of the previous trial re- 
sults. Whether or not this assumption holds does not af- 
fect the validity of the p- value computed. The confidence 
gain of the n'th trial may be defined as log 2 R n {x n ). Its 
expected value given that X n is distributed according to 
q is 
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q(x n ) log 2 R 



(6) 



Before the n'th trial, the protocol attempts to maximize 
this expected confidence gain. Since q is not known, it 
is empirically estimated based on the first (n — 1) trials. 
Expanding R n according to Eq. ^ yields the following 
estimate of the expected confidence gain of the n'th trial: 



G n (u>) 



n-l 
fe=l 



log 2 (^ • r(x fc )). 



(7) 



The protocol thus determines R n by maximizing G n {oj) 
over u>, that is, R n = r • argmax w G ra (o;). Note that, 
unlike the full PBR protocol, the simplified PBR protocol 
does not require explicitly optimizing over all LR models. 
Computing argmax l4j G n (u;) requires optimizing a convex 
objective function over an M-dimensional convex space, 
where the evaluation of the objective function involves a 
sum of (n— 1) terms. In our implementation, we apply the 



expectation-maximization (EM) algorithm [TB] to solve 
this problem. 

The performance of the simplified PBR protocol de- 
pends on the relationship between the actual distribution 
of trial results and the set of standardized Bell functions 
used. If the results are independent and identically dis- 
tributed according to a known distribution that violates 
LR, then there exists an optimal Bell inequality that can 
be derived from the optimal PBR as found by the full 
PBR protocol. (Here, optimality refers to the optimality 
of the gain rate achieved by the protocol; see Ref. [5].) 
If the optimal PBR is included in the convex set of stan- 
dardized Bell functions, the confidence-gain rate achieved 
by the simplified PBR protocol is optimal. But since the 
actual distribution is unknown before an experiment, the 
above assumption may not hold without making the di- 
mension of the set of convex combinations in Eq. ([5| im- 
practically large. Thus, before an experiment, it is im- 
portant to choose a relevant (and preferably small) set of 
standardized Bell functions. Below we show that it helps 
to include more than just the obvious Bell functions. 

The performance of the simplified PBR protocol can 
be compared with that of the martingale-based proto- 
col [5J[I0], the only valid non-PBR protocol considered so 
far. The martingale-based protocol uses a Bell inequality 
(I(X)) < B with a bounded Bell function, chosen before 
an experiment. After the experiment, the mean of the 
Bell function is estimated as / = -k J^ n=1 1(x n ). To ob- 
tain a p-value, the protocol uses / as the test statistic. 
If I > B, the bounds on the Bell function imply that a 
p-value can be computed according to 
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Pn 



a-B 
a-I 



B-b 
I-b 



N 



(8) 



where a — sup x I(x) and b — mi x I(x). This p-value 
expression is based on a version of Hoeffding's bound in 
Ref. [17] . which improves the ones given in Refs. [51I51[TU]. 
The derivation of Eq. Q is explained in the supplemen- 
tal material (SM). In the SM, we also show that the 
simplified PBR protocol using the same Bell inequality, 
together with the default trivial Bell function r = 1, 
achieves a gain rate at least as high as the gain rate 
achieved by the martingale-based protocol. These two 
gain rates are equal to each other if and only if the ex- 
perimental range of the function / is contained in the set 
{a, 6}. 

Computational resource costs. — Of the available pro- 
tocols for computing p-values, the martingale-based one 
is the least resource-intensive and simplest to apply. It 
requires computing only an estimate of the mean of the 
Bell function, which involves a sum of N terms. The 
motivation for applying a PBR protocol is that it can 
adapt to changes in the experimental results' distribu- 
tion and find better Bell inequalities with respect to the 
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observed data, both of which are difficult to do before an 
experiment. 

For quantifying the resource cost of the simplified PBR 
protocol, we assume that the Bell functions used can be 
evaluated in constant time given the values of the argu- 
ments. This assumption is realistic for many Bell func- 
tions, as their values are determined by concise formulas 
derived from theory. Alternatively, these functions can 
be preprocessed as a table stored in random-access mem- 
ory; we do not include preprocessing time in our analysis. 
We also assume that an optimization is performed by al- 
gorithms whose resource costs are determined primarily 
by the complexity G of evaluating the objective function 
and the dimension D of the convex search space. In par- 
ticular, we do not account for algorithm-specific resource 
costs such as the number of iterations required to achieve 
sufficient precision for our purpose. Also, we do not ac- 
count for the complexity of enforcing convex constraints. 
This is motivated by the observation that there is no ad- 
ditional overhead for enforcing convex constraints in the 
EM algorithm used in our implementation. Given our 
assumptions and from Eq. (J7|, the complexity of evalu- 
ating the objective function in the optimization required 
for constructing the PBR R n is C — 0(nM), which is 
the product of the number of terms in the sum and the 
number of Bell-function evaluations underneath the loga- 
rithm. The dimension of the search space is D = O(M), 
the size of u> in Eq. Consequently, unlike the full 

PBR protocol (see the details in the SM), the numbers 
of parties, settings, and outcomes are not limiting factors. 
In this sense, the simplified PBR protocol is efficient for 
any experimental configuration. 

In our discussion so far, we have assumed that each 
PBR protocol updates the PBR before each trial. In 
practice, the PBR is updated only for a block of trial 
results at a time (see Ref. [8]), thus limiting PBR com- 
putations to when enough new information has been ob- 
tained, thereby reducing the resource cost. 

Protocol comparison. — We begin by comparing the 
confidence-gain rates achieved by different protocols 
for experimental configurations designed to violate 
the Collins-Gisin-Lindcn-Massar-Popescu (CGLMP) in- 
equality [IS]. To test the CGLMP inequality, there are 
two parties, and each of them performs one of two possi- 
ble measurements with d outcomes at each trial. This is 
an example where the full PBR protocol is impractical for 
large d. For this example and the one below, we assume 
that at each trial each party's measurement setting is cho- 
sen uniformly randomly. The CGLMP inequality can be 
written as (Id(X)) < 2, where the function Id takes d dif- 
ferent values. The gain rates G mar t and G s pbr, achieved 
by the martingale-based and simplified PBR protocols, 
are shown in Fig. [T] Here the simplified PBR protocol 
uses only the CGLMP inequality. This figure illustrates 
that G s pBR is higher than G mar t when d > 2. 

The optimal gain rate S q is achieved by the full PBR 
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FIG. 1: Confidence-gain rates in the test of the CGLMP in- 
equality (Id(X)) < 2. Here, we use the quantum state and 
measurement settings of Ref. [15) . Eqs. (15) and (9), respec- 
tively. 



protocol and can be computed as the minimum KL di- 
vergence from the experimental probability distribution 
to any LR model [5D] ■ For the results of Fig. [I] we find 
that the gain rates G s pbr are numerically indistinguish- 
able from S q when d < 13. For the case d > 13, it is 
difficult to compute S q due to the large dimension of the 
probability space over all possible LR models. For the 
tests studied in Fig. [TJ we conjecture that G s pbr = S q . 
In general we cannot guarantee that G s pbr is optimal. 

Next, we compare the performance of the simplified 
PBR protocol when using different numbers of Bell in- 
equalities. The experimental configuration considered is 
for a test of the CHSH inequality ([I]) using an unbalanced 
Bell state \^{6)) = cos(0)|OO) +sin(0)|ll). For compar- 
ison, we consider the simplified PBR protocol with the 
CHSH inequality alone or in conjunction with additional, 
seemingly trivial Bell inequalities such as those derived 
from no-signaling conditions. With Bell functions cor- 
responding to no-signaling conditions, the gain rates are 
improved, as shown in Fig. [2] 

In the SM, we show how the p-values computed by 
different protocols behave in a simulated experiment as 
functions of the number of trials. 

Extensions. — To compute a p-value, the simplified 
PBR protocol uses a set of linear inequalities that are 
satisfied by the predictions of a null hypothesis before 
each trial in an experiment. Besides tests of LR, there 
are many other types of tests based on linear witnesses, 
such as tests for entanglement [TTJ [T5] and system dimen- 
sionality above a given bound [3TJ [55] . In any test based 
on linear witnesses, such a witness can be expressed as 
(W(X)) < B, where W is a real-valued function and X 
is the random variable from which a trial result x is sam- 
pled. The result x consists of all choices made at each 
trial, such as choices of states and measurement settings, 
and the outcomes observed under these choices. Here, 
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FIG. 2: Confidence-gain rates in the test of LR with an unbal- 
anced Bell state \ip(6)). The measurement settings are chosen 
to maximize the violation of the CHSH inequality Q given 
the state \ip{0)). The gain rates achieved by the simplified 
PBR protocol using the CHSH inequality are shown as circles 
(o), while the gain rates by the same protocol using the CHSH 
inequality together with no-signaling conditions are shown as 
crosses (+). 

we assume that the choices are made randomly accord- 
ing to a known probability distribution at each trial, so 
that a witness (W(X)) < B is satisfied before each trial 
assuming the null hypothesis. As for Bell functions, if a 
witness W is lower-bounded it can be standardized. The 
simplified PBR protocol can then be applied with any set 
of standardized witnesses, as we did in a test of LR. 

We thank Kevin Coakley for helpful comments and 
discussions. This paper is a contribution of the National 
Institute of Standards and Technology and is not subject 
to U.S. copyright. 

Supplemental material 
Computational resource comparison 

In this section, we compare the computational re- 
sources required by the simplified and full PBR protocols 
in an experimental test of LR. 

We consider an experimental configuration involving I 
parties where each party has s measurement settings and 
each local measurement has d outcomes. (The compari- 
son below is readily extended to more general configura- 
tions.) We suppose that the joint-setting distribution is 
uniform. Then, the number of possible results (measure- 
ment settings and outcomes of all parties) at a trial is 
K = (ds) 1 . Since a deterministic LR model specifies the 
exact outcome for each local measurement of each party 
at a trial, there are H — d ls many such models. A general 
LR model is a convex combination of deterministic LR 
models, so the number of free parameters characterizing 
a general LR model is (H — 1). 



Let the total number of trials in an experimental test 
of LR be N. We assume that each PBR protocol sets 
the initial value of the PBR to R\ — 1 and updates 
the PBR R n before each trial n (n > 1). (In practice 
this is unnecessary; see the appendix of Ref. [8 .) For 
updating the PBR, each PBR protocol needs to opti- 
mize a convex objective function over a convex space. 
The complexity of this optimization problem can be de- 
scribed in terms of variables that are functions of the 
parameters n, I, s, and d characterizing the input data 
size. (Note that the stored size of the first n trial results 
is 0{n\og{K)) = 0(nl(log(d) + log(s))).) We need to 
quantify the resource cost of implementing each protocol 
in terms of these parameters. 

The complexity of the optimization problem solved be- 
fore each trial can be parametrized by the complexity of 
the convex search space, the complexity of evaluating the 
objective function, and the precision needed for comput- 
ing a high-quality p-value for rejecting LR. We assume 
that the simplified and full PBR protocols use generic 
iterative optimization algorithms whose implementation 
complexities as functions of these parameters are asymp- 
totically the same. We also assume that the complexity 
of the convex search space is dominated by its dimen- 
sion. In particular, we do not account for the complex- 
ity of enforcing convex constraints. For quantifying the 
complexity of evaluating the objective function, we as- 
sume that the Bell functions used can be evaluated in 
constant time given any trial result, as explained in the 
main text. Also, we assume that determining whether 
or not an arbitrary trial result x happens according to 
a deterministic LR model takes constant time. (Strictly 
speaking, the time taken for such a determination process 
is proportional to the number of parties I.) The precision 
needed affects the number of iterations required by an al- 
gorithm to find a numerical solution. It affects only the 
quality of the p-value computed by a protocol, but not 
its validity. (For the EM algorithm [TH] used, see The- 
orem 4 of Ref. [TB] and the appendix of Ref. [H] for the 
effects of the precision parameters in the simplified and 
full PBR protocols, respectively.) We assume that the 
precision parameters in both protocols are set to be the 
same, and we do not account for the number of iterations 
required to achieve the specified precision. Therefore, for 
the purpose of comparing the computational resources 
required by the simplified and full PBR protocols, we fo- 
cus on comparing the dimensions D of the convex search 
spaces and the complexities C of evaluating the objective 
functions in the optimization problems solved by the two 
protocols before each trial. 

We first consider the simplified PBR protocol. Given a 
set of M Bell inequalities, this protocol sets R n = w„ • r, 
where the size of w n is M, r is defined before Eq. ^ in 
the main text, and u>„ is chosen to maximize the esti- 
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mated confidence gain 



n-l 
k=l 



log 2 (u; • r(x k )) = fn(x)log 2 (u}-r(x)), 



x:f n (x)^0 



(A.9) 

where f n (x) is the empirical frequency of x before the 
n'th trial. Note that, in the right-hand side of Eq. (A.9 1, 



the sum is taken over only the results x already observed 
in the previous trials. 

For the maximization of Eq. (A.9), the dimension of 
the convex search space is M. The evaluation of the 
objective function can use the left-hand or right-hand 
side of Eq. (A.9), whichever has fewer terms. Thus it 



involves a sum of at most min(n — terms where 

each term requires computing a convex combination of 
M Bell-function values. Hence, for updating the PBR 
R n before the n'th trial, the complexity of evaluating 
the objective function is C s pbr = 0(min(nM, KM)) = 
0(min(nM,(ds) M)), and the dimension of the search 
space is -D s pbr = 0(M). Therefore, if any of the config- 
uration parameters I, s, or d is large, C s pbr and D s pbr 
are independent of these parameters, and so the simpli- 
fied PBR protocol can be applied efficiently. 

The full PBR protocol j5] computes R n in two steps. 
First, the protocol estimates the probability q(x) of the 
result x to be observed at the next trial. This es- 
timate can be obtained in different ways. The sim- 
plest is to let q(x) be the empirical frequency f n (x) of 
x over the previous (n — 1) trials. However, one can 
consider additional constraints such as the known joint- 
setting distribution and no-signaling conditions. Thus, 
in Ref. |8] we suggested maximizing the log-likelihood 
function L(q') oc J2 X f n (x) log 2 (g'(x)), subject to these 
constraints, and we observed that this can improve the 
quality of the p- value computed. Since this maximization 
is not a resource bottleneck, we do not consider its com- 
plexity in the comparison. Second, we find the LR model 
p closest to the estimated distribution q by minimizing 
the KL divergence |15j from q to an LR model plr 



D K h{q\PhR) = v( x ) lo S2 



Plr(x) 



(A.IO) 



The full PBR protocol then sets R n {x n ) — q(x n )/p(x n ). 

For the minimization of Eq. (A.IO), the dimension of 
the convex search space is H . The evaluation of the ob- 
jective function involves a sum of K terms where each 
term requires computing Plr(^) according to a convex 
combination of H deterministic LR models. Hence, for 
updating the PBR R n before the n'th trial, the com- 
plexity of evaluating the objective function is CfPBR = 
O(KH) — 0(d'( s+1 ) s l ), and the dimension of the search 
space is Apbr = 0(H) = 0(d ls ). While Cfp B R and 
DfPBR are polynomial in d, they are exponential in each 
of I and s. Therefore, the full PBR protocol is not effi- 



cient with respect to these configuration parameters. 

Before applying the simplified PBR protocol, one 
chooses a relevant and preferably small set of Bell in- 
equalities. In many cases of interest, I, s, or d is large, and 
so is H = d sl . For example, in field-quadrature measure- 
ments d is fundamentally infinite. Hence, M, the number 
of Bell inequalities used in the simplified PBR protocol, 
is in general much smaller than H, the number of deter- 
ministic LR models considered in the full PBR protocol. 
The complexities show that for such cases, the simpli- 
fied PBR protocol is substantially less resource-intensive 
than the full PBR protocol. 



The martingale-based protocol's p-value 

Consider a Bell inequality (I(X)) < B with a Bell 
function / whose range is included in the interval [b, a] , 
where b < a. An experimental test yields an estimate 
7 = 4? J2n=i I( x n) °f the mean of I, where x\, . . . , xn 
are the trial results. 

Suppose that the n'th trial result x n is distributed ac- 
cording to a random variable Alr^ satisfying LR. In 
this case, the random variable from which I is sam- 
pled is Jlr = jf J2n=i I(Xlr,7i)- The sequence M n = 
^^ =1 (/(ALR,fe) — B) is a super-martingale, as shown in 
Refs. [10]. Thus, for t > 0, the probability 



Prob LR (M N > Nt) < 

a- B 
a-B-t 



B-b 
B + t-b 



N 



(A.11) 



Theorem 6.1 
= Prob LR (M N 



of 

> 



The inequality (A.ll) follows from 
Ref. [23]. Since Prob LR (7 LR > I) - 
N(I — B)), from the above inequality (A.ll ) we get the 
p- value of Eq. ([8]) in the main text. 

Note that, although Theorem 6.1 of Ref. 23 is stated 
for a martingale, the same result and its proof also ap- 
ply to a super-martingale. The same bound is also de- 
rived in Theorem 1 of Ref. [17] for a sum of indepen- 
dent random variables. From Refs. [T71H3], we can see 
that the bound in Eq. (A.ll) is tighter than bounds of 
ProbLR (Mff > Nt) used in previous works |H1 HH] and 
derived from Azuma's inequality (23] [24] . 



Proof Of Gmart < GsPBR 

We suppose that the martingale-based protocol uses a 
Bell inequality (I(X)) < B with a bounded Bell function 
such that b < I(x) < a for all x. Also, we suppose 
that the simplified PBR protocol uses the standardized 
form of this Bell inequality together with the trivial Bell 
function r = 1. 
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Let the experimental probability of observing the re- 
sult x be q(x). The experimental mean of I is I q = 
J q{x)I(x)dx. If I q > B, then from Eqs. ^ and Q in 
the main text we get the gain rate 



G n 



- b 



_^ + v^ log2 v^ 

a - B a - b B — b 



. a - I{x) a- I q 

i( x ) ( — : — — lo S2 



- b 



a-B 



I{x) - b 
a — b 



log: 



Ig-b 
B-b 



dx. 



(A.12) 



Here, we use the fact that the experimental estimate / 
approaches I q as N — > oo. By the concavity of log 2 (a:) 
and some algebra, we get that the gain rate G mart satisfies 
the inequality 



G,, 



< 



iog 2 



a — I{x) a — I q I(x) 
a — b a — B 
I(x) - b 



bI «-^dx 



B-b 



a-b B-b 
1 ljo) dx, 

(A.13) 



Gspbr = max 

Q<u<l 



q{x) log 



I{x) - b 



\ — ijj \ dx 



Eq. (A.13) with Eq. (A.14) 




2000 



4000 



6000 



8000 



10000 



where < cjn = Iq p < 1- 
— u . . a— is 

From Eqs. pi and Q in the main text and according 
to the design of the PBRs by the simplified PBR protocol 
(as explained in the main text), the gain rate achieved 
by this protocol is 



B-b 

(A.14) 

Here, we use the fact that the empirical frequency /at (x) 
approaches the experimental probability q(x) as N — >• oo. 
The inequality G ma rt < G s pbr follows from comparing 



By considering the condition for equality in Eq. (A.13 1, 
we can show that G mar t = G s pbr if and only if q(x) = 
whenever b < I(x) < a. For this it suffices to note that 



log 2 (a;) is strictly concave, so equality holds in Eq. (A.13) 
if and only if I{x) = a or b whenever q{x) ^ 0. 



Behavior of the protocols for finite data 

Here we consider the behavior of each protocol given 
a finite amount of experimental data. We simulate the 
test of the CGLMP inequality (h(X)) < 2 [18] with the 
quantum state and measurement settings of Ref. [19], 
Eqs. (15) and (9) (with d = 3), respectively. We as- 
sume that at each trial each party's measurement setting 
is chosen uniformly randomly. The protocols' gain rates 
are G mar t = 0.0565 and G s pbr = 0.0675, while the op- 
timal gain rate S q achieved by the full PBR protocol is 
numerically indistinguishable from G s pbr. For comput- 
ing G s pBR, the simplified PBR protocol uses the stan- 
dardized CGLMP inequality and the trivial Bell function 



FIG. 3: An example of running log-p- values as functions of 
the number of trials n in a test of the CGLMP inequality. 
The dashed and solid lines are the asymptotic lines for log- 
p-values based on gain rates according to the (full or sim- 
plified) PBR protocol and the martingale-based protocol, re- 
spectively. Repetitions of this Monte Carlo simulation show 
similar behavior. 



The results from 10, 000 successive trials are recorded. 
Fig.|3]shows the (negative) log-p- values computed for the 
first n results from a simulated sequence of trials as func- 
tions of n. The asymptotic lines for log-p- values, given by 
the products of n and the respective gain rates achieved 
by different protocols, are also shown in Fig. [3] 

For the simulation shown in Fig. [3] we update the 
PBRs and log-p-values only after every block including 
154 successive trials. (See our previous work [5] for a dis- 
cussion of the block-size choice and related issues.) This 
improves the efficiency of a PBR protocol. It also mit- 
igates the offset of the computed log-p-values from the 
asymptotic line. This offset is due to an initial transient 
where the relevant features of the experimental distribu- 
tion are being learned. The learning offset can be re- 
moved if, before an experiment, we have a good estimate 
of the experimental results' distribution. Such an esti- 
mate could be based on (quantum or otherwise) theory 
or previous experiments. 

The PBR protocols provide better results than the 
martingale-based protocol. However, the PBR log-p- 
values show learning offsets from the asymptotic line. 
Our results show that the simplified PBR log-p-values 
have a smaller learning offset than the full PBR log-p- 
values in each of 30 independent simulations performed. 
The reason is that the simplified PBR protocol needs to 
infer a much smaller number of parameters for construct- 
ing the PBRs. 

In the above example, the simplified PBR protocol uses 
only two Bell functions. Given a prescient choice of Bell 
functions, this is sufficient for computing asymptotically 
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optimal p-values. But in general, more Bell functions are 
needed for computing a high-quality p-value. However, 
this involves inferring more parameters and thus requires 
more trials before a good inference can be obtained. As 
a result, the learning offset is expected to increase when 
using more Bell functions. One way to mitigate this prob- 
lem may be to increase the number of Bell functions used 
over time, adding new Bell functions only when there are 
enough trials for reliable inference of the additional pa- 
rameters. 
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